Modelling Physical Activity in Virtual Reality Games

This thesis was inspired by the possibility that virtual reality (VR) games, which are designed primarily to be fun, could also provide exercise. It aimed to gain insights about this by exploring whether people can gain beneficial levels of exercise while playing VR games and how they might use VR games for exercise over several weeks. Furthermore, this work also focuses on how the level of physical activity that can be captured during gameplay and how a long-term user model can be created for individual players, as a foundation for supporting the user in gaining personal informatics insights about their exertion as well as being used for personalisation and external recommendation for VR games. The key contributions of this research are: • The first study of a diverse set of commercial VR games to gain insights about the level of actual and perceived exertion players have. • The first long-term study of VR games in a sedentary workplace to gain insights about the ways people utilise it and the levels of exertion they gain. • Based on reflections on the above studies, this thesis presents a framework and guidelines for designing physical activity VR games. • The systematic creation of a user model for representing a person’s long-term fitness and their VR gameplay, exertion and preferences. • A study of the ways that people can scrutinise their long-term personal informatics user model of exertion from VR game play and incidental walking. These contributions provide a foundation for future researchers and industry practitioners to design VR games that provide beneficial levels of exertion and allow people to gain insights into the relative contribution of the exercise from gameplay

The Effect of Luminance on Depth Perception in Augmented Reality Guided Laparoscopic Surgery

Depth perception is a major issue in surgical augmented reality (AR) with limited research conducted in this scientific area. This study establishes a relationship between luminance and depth perception. This can be used to improve visualisation design for AR overlay in laparoscopic surgery, providing surgeons a more accurate perception of the anatomy intraoperatively. Two experiments were conducted to determine this relationship. First, an online study with 59 participants from the general public, and second, an in-person study with 10 surgeons as participants. We developed 2 open-source software tools utilising SciKit-Surgery libraries to enable these studies and any future research. Our findings demonstrate that the higher the relative luminance, the closer a structure is perceived to the operating camera. Furthermore, the higher the luminance contrast between the two structures, the higher the depth distance perceived. The quantitative results from both experiments are in agreement, indicating that online recruitment of the general public can be helpful in similar studies. An observation made by the surgeons from the in-person study was that the light source used in laparoscopic surgery plays a role in depth perception. This is due to its varying positioning and brightness which could affect the perception of the overlaid AR. We found that luminance directly correlates with depth perception for both surgeons and the general public, regardless of other depth cues. Future research may focus on comparing different colours used in surgical AR and using a mock operating room (OR) with varying light sources and positions

Designing Digital COVID-19 Screening: Insights and Deliberations

Due to the global COVID-19 pandemic, public health control and screening measures have been introduced at healthcare facilities, including those housing our most vulnerable populations. These warning measures situated at hospital entrances are presently labour-intensive, requiring additional staff to conduct manual temperature checks and risk-assessment questionnaires of every individual entering the premises. To make this process more efficient, we present eGate, a digital COVID-19 health-screening smart Internet of Things system deployed at multiple entry points around a children's hospital. This paper reports on design insights based on the experiences of concierge screening staff stationed alongside the eGate system. Our work contributes towards social-technical deliberations on how to improve design and deploy of digital health-screening systems in hospitals. It specifically outlines a series of design recommendations for future health screening interventions, key considerations relevant to digital screening control systems and their implementation, and the plausible effects on the staff who work alongside them

Enhanced Surgeons: Understanding the Design of Augmented Reality Instructions for Keyhole Surgery

It is important to understand how to design AR content for surgical contexts to mitigate the risk of distracting the surgeons. In this work, we test information overlays for AR guidance during keyhole surgery. We performed a preliminary evaluation of a prototype, focusing on the effects of colour, opacity, and information representation. Our work contributes insights into the design of AR guidance in surgery settings and a foundation for future research on visualisation design for surgical AR

Towards an Inclusive and Accessible Metaverse

The push towards a Metaverse is growing, with companies such as Meta developing their own interpretation of what it should look like. The Metaverse at its conceptual core promises to remove boundaries and borders, becoming a decentralised entity for everyone to use - forming a digital virtual layer over our own "real"world. However, creation of a Metaverse or "new world"presents the opportunity to create one which is inclusive and accessible to all. This challenge is explored and discussed in this workshop, with an aim of understanding how to create a Metaverse which is open and inclusive to people with physical and intellectual disabilities, and how interactions can be designed in a way to minimise disadvantage. The key outcomes of this workshop outline new opportunities for improving accessibility in the Metaverse, methodologies for designing and evaluating accessibility, and key considerations for designing accessible Metaverse environments and interactions

The value of Augmented Reality in surgery — A usability study on laparoscopic liver surgery

Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions — a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future

Augmenting Cities and Architecture with Immersive Technologies

Immersive technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) have the potential to augment experiences within cities and the process of designing architecture. However, more work is needed to understand specific applications within these areas and how they can be designed. Therefore, the main aim of the workshop is to discuss and ideate use-cases for creating situated immersive AR, VR, and MR applications for the purpose of making cities more engaging and to help design the cities of the future

Nurse, Give Me the News! Understanding Support for and Opposition to a COVID-19 Health Screening System

Helping the sick and protecting the vulnerable has long been the credo of the health profession. In response to the coronavirus-disease-2019 (COVID-19 pandemic), hospitals and healthcare institutions have rapidly employed public health measures to mitigate patient and staff infection. This paper investigates staff and visitor responses to the COVID-19 eGate health screening system; a self-service technology (SST) which aims to protect health care workers and facilities from COVID-19. Our study evaluates the in situ deployment of the eGate, and employs a System Usability Scale (SUS) and questionnaire (n = 220) to understand staff and visitor’s acceptance of the eGate. In detailing the themes relevant to those who advocate for the system and those who oppose it, we contribute towards a more detailed understanding of the use and non-use of health-screening SSTs. We conclude with a series of considerations for the design of future interactive screening systems within hospitals

Augmenting Cities and Architecture with immersive Technologies: Conference Workshop Proceedings

Augmented reality (AR) has many domains it can be applied to, such as collaboration, heads up display navigation and operating heavy machinery. Recently, AR has shown promise at engaging communities and enhancingindividual experiences within cities. While science fiction has depicted future cities imbued with holograms, recent work has shown that AR can have more engaging applications, in areas like community engagement, personalised digital signage, and visualisation. On a consumer level, AR is most common on smartphones, particularly after the release of ARKit and ARCore - improving the functionality. Smartphone AR apps have primarily consisted of location-aware smartphone augmented reality (AR) apps, like Pokemon Go – bringing people outside and having ripple effects on the physical space through physical signage. Smart glasses are also becoming more accessible and enable the possibility to naturally integrate virtual content into our daily lives. While immersive technologies are advancing and gradually becoming accessible, further understanding is needed of their application and the potential benefits they bring to cities, communities, and individuals. This also brings up the question of how the virtual and physical spaces co-exist creating an augmented space.This year we narrowed the focus from the broader cities to how such immersive technologies (like AR and VR) can be used to improve people’s wellbeing within a city environment (such as taking someone to another world, gamifying the city, or playfully reminding people to take the stairs). For this workshop it was more important than ever to consider people’s wellbeing in cities as global pandemics and environmental issues change the way people operate in cities. This workshophas been run at MAB 2018 and ISMAR 2019, where it has received attendees from a range of different backgrounds. Many interesting insights have been brought up during the workshop around security, privacy, and use cases. After ISMAR, we guest edited a focus section in a special issue of IxDA (Interaction Design and Architectures Journal)

Preface:Augmenting Space: The role of immersive technologies in future cities

As digital technologies are advancing, contemporary interactions within cities are beginning to emerge. These interactions are commonly enabled through sensors to implicitly automate manual processes, such as turning on lights or walking up stairs. However, cities were not necessarily built from the ground up to be smart, rather they are gradually becoming smarter over time as technology becomes more extensible and embedded within them [1, 2]. These digital technologies create information layers that exist over the physical space, resulting in the space being filled with dynamically changing information, thus augmenting the space [3]. Augmented reality (AR) is one such technology that has recently seen a lot of development in this area and is only now starting to become more viable as hardware and computer vision algorithms have caught up. Films such as Minority Report (2002), Ghost in the Shell (2016), and Blade Runner (2017) have predicted AR’s future emergence in public spaces and cities [4, 5, 6]. These films featured AR advertising and information holograms in public spaces, enabled by smart contact lenses and holograms. Currently however, AR has been introduced to public spaces in a number of interesting ways. For instance, Pokemon GO became a global phenomenon which resulted in people physically playing the game in urban spaces and caused ripple effects on the physical spaces people were playing in [7, 8, 9]. Recent work has also shown that AR can have more engaging applications, in areas such as community engagement [10], personalised digital signage [11], in-situ visualisations [12], cultural heritage [13], and remote collaboration [14, 15]. On a consumer level, AR is most common on smartphones, particularly after the release of ARKit and ARCore - improving the functionality. AR smart glasses are also becoming more accessible and bring with them the possibility of more natural integration of virtual content into our daily lives. For instance, the Microsoft Hololens contains an array of sensor technologies giving it a sense of depth which allows it to place objects naturally in physical space. It has been successfully applied by planners to visualise underlying parts of the city in-situ [16, 17]. While AR is becoming more advanced, accessible, and has demonstrated potential, more knowledge is needed around the key benefits it will bring to cities and how it will change our interactions with the urban environment. Additionally, the use of such technologies raises the question of how the virtual and physical spaces can co-exist - creating an augmented space [3]. To address this gap in knowledge, this focus section builds on from our initial workshops [18] at Media Architecture Biennale (MAB) 2018 and IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2019 with the goal of bringing together researchers to explore the applications of AR and other immersive technologies, such as Virtual Reality (VR) and Mixed Reality (MR), within the context of enhancing architecture, public spaces and cities. In response, seven papers were selected for the focus section through a single-blinded peer-review process with at least two reviewers per paper. The papers showcase the applicability of various methods in a wide variety of different use cases, providing insights into current possibilities and Interaction Design and Architecture(s) Journal - IxD&A, N.48, 2021, pp. 5 - 7 5 challenges for digital technologies and the ways in which they can augment cities around the world. References 1. Tomitsch, M. (2018). Making cities smarter: designing interactive urban applications. Jovis. 2. Parker, C., Tomitsch, M., & Fredericks, J. (2021). Smart engagement for smart cities: Design patterns for digitally augmented, situated community engagement. In Shaping Smart for Better Cities (pp. 177-200). Academic Press. 3. Manovich, L. The poetics of augmented space. Visual Communication 5, 2 (2006), 219–240. 4. Arthur, C. Why minority report was spot on, 2010. 5. Jones, E. How soon could ghost in the shell’s fantasy technology become a reality?, 2017. 6. Ghahramani, A. What “blade runner 2049” gets right (and wrong) about ar, 2017. 7. Adlakha, D., Marquet, O., Hipp, J. A., and Tully, M. A. Pokemon go or pokemon gone: How can cities respond to trends in technology linking people and space? Cities & Health 1, 1 (2017), 89–94. 8. Colley, A., Thebault-Spieker, J., Lin, A. Y., Degraen, D., Fischman, B., Hakkila, J., Kuehl, K., Nisi, V., Nunes, N. J., Wenig, N., et al. The geography of pokemon go: beneficial and problematic effects on places and movement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (2017), ACM, pp. 1179–1192. 9. Boulos, M. N. K., Lu, Z., Guerrero, P., Jennett, C., and Steed, A. From urban planning and emergency training to pokemon go: applications of virtual reality gis (vrgis) and augmented reality gis (argis) in personal, public and environmental health, 2017. 10. Fredericks, J., Hespanhol, L., Parker, C., Zhou, D., and Tomitsch, M. Blending pop-up urbanism and participatory technologies: Challenges and opportunities for inclusive city making. City, Culture and Society (2017). 11. Parker, C., Kay, J., Baldauf, M., and Tomitsch, M. Design implications for interacting with personalised public displays through mobile augmented reality. In Proceedings of the 5th ACM International Symposium on Pervasive Displays (2016), ACM, pp. 52–58. 12. Lee, G. A., Du¨nser, A., Kim, S., and Billinghurst, M. Cityviewar: A mobile outdoor ar application for city visualization. In Mixed and Augmented Reality (ISMAR-AMH), 2012 IEEE International Symposium on (2012), IEEE, pp. 57–64. 13. Park, H., Kim, E., Kim, H., Shin, J. E., Kim, J., Kim, K., & Woo, W. (2018, July). K-culture time machine: A mobile ar experience platform for korean cultural heritage sites. In International Conference on Human Interface and the Management of Information (pp. 167- 180). Springer, Cham. 14. Lee, Y., Masai, K., Kunze, K., Sugimoto, M., and Billinghurst, M. A remote collaboration system with empathy glasses. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (Sep. 2016), pp. 342–343 Interaction Design and Architecture(s) Journal - IxD&A, N.48, 2021, pp. 5 - 7 6 15. Jenek, W. Architecture with immersive technologies: Next generation architects and clients. 16. Hockett, P., and Ingleby, T. Augmented reality with hololens: Experiential architectures embedded in the real world. arXiv preprint arXiv:1610.04281 (2016). 17. Zhang, L., Chen, S., Dong, H., and El Saddik, A. Visualizing toronto city data with hololens: Using augmented reality for a city model. IEEE Consumer Electronics Magazine 7, 3 (2018), 73–80. 18. Parker, C., Jenek, W., Yoo, S., and Lee, Y. Augmenting cities and architecture with immersive technologies. In Proceedings of the 4th Media Architecture Biennale Conference (2018), ACM, pp. 174–177